Computational Audiovisual Scene Analysis

Yan R. Computational Audiovisual Scene Analysis. Bielefeld: Universitätsbibliothek Bielefeld; 2014.In most real-world situations, a robot is interacting with multiple people. In this case, understanding of the dialogs is essential. However, dialog scene analysis is missing in most existing systems of human-robot interaction. In such systems, only one speaker can talk with the robot or each speaker wears an attached microphone or a headset. The target of Computational AudioVisual Scene Analysis (CAVSA) is therefore making dialogs between humans and robots more natural and flexible. The CAVSA system is able to learn how many speakers are in the scenario, where the speakers are and who is currently speaking. CAVSA is a challenging task due to the complexity of dialogue scenarios. First, speakers are unknown in advance, thus a database for training high-level features beforehand to recognize faces or voices is not available. Second, people can dynamically come into and leave the scene, may move all the time and even change their locations outside the camera field of view. Third, the robot can not see all the people at the same time due to limited camera field of view and head movements. Moreover, a sound could be related to a person who stands outside the camera field of view and has never been seen. I will show that the CAVSA system is able to assign words to corresponding speakers. A speaker is recognized again when he leaves and enters the scene, or changes his position even with a newly appearing person

Identification and validation of a novel cuproptosis-related gene signature in multiple myeloma

Background: Cuproptosis is a newly identified unique copper-triggered modality of mitochondrial cell death, distinct from known death mechanisms such as necroptosis, pyroptosis, and ferroptosis. Multiple myeloma (MM) is a hematologic neoplasm characterized by the malignant proliferation of plasma cells. In the development of MM, almost all patients undergo a relatively benign course from monoclonal gammopathy of undetermined significance (MGUS) to smoldering myeloma (SMM), which further progresses to active myeloma. However, the prognostic value of cuproptosis in MM remains unknown.Method: In this study, we systematically investigated the genetic variants, expression patterns, and prognostic value of cuproptosis-related genes (CRGs) in MM. CRG scores derived from the prognostic model were used to perform the risk stratification of MM patients. We then explored their differences in clinical characteristics and immune patterns and assessed their value in prognosis prediction and treatment response. Nomograms were also developed to improve predictive accuracy and clinical applicability. Finally, we collected MM cell lines and patient samples to validate marker gene expression by quantitative real-time PCR (qRT-PCR).Results: The evolution from MGUS and SMM to MM was also accompanied by differences in the CRG expression profile. Then, a well-performing cuproptosis-related risk model was developed to predict prognosis in MM and was validated in two external cohorts. The high-risk group exhibited higher clinical risk indicators. Cox regression analyses showed that the model was an independent prognostic predictor in MM. Patients in the high-risk group had significantly lower survival rates than those in the low-risk group (p < 0.001). Meanwhile, CRG scores were significantly correlated with immune infiltration, stemness index and immunotherapy sensitivity. We further revealed the close association between CRG scores and mitochondrial metabolism. Subsequently, the prediction nomogram showed good predictive power and calibration. Finally, the prognostic CRGs were further validated by qRT-PCR in vitro.Conclusion: CRGs were closely related to the immune pattern and self-renewal biology of cancer cells in MM. This prognostic model provided a new perspective for the risk stratification and treatment response prediction of MM patients

A novel glycolysis-related gene signature for predicting the prognosis of multiple myeloma

Background: Metabolic reprogramming is an important hallmark of cancer. Glycolysis provides the conditions on which multiple myeloma (MM) thrives. Due to MM’s great heterogeneity and incurability, risk assessment and treatment choices are still difficult.Method: We constructed a glycolysis-related prognostic model by Least absolute shrinkage and selection operator (LASSO) Cox regression analysis. It was validated in two independent external cohorts, cell lines, and our clinical specimens. The model was also explored for its biological properties, immune microenvironment, and therapeutic response including immunotherapy. Finally, multiple metrics were combined to construct a nomogram to assist in personalized prediction of survival outcomes.Results: A wide range of variants and heterogeneous expression profiles of glycolysis-related genes were observed in MM. The prognostic model behaved well in differentiating between populations with various prognoses and proved to be an independent prognostic factor. This prognostic signature closely coordinated with multiple malignant features such as high-risk clinical features, immune dysfunction, stem cell-like features, cancer-related pathways, which was associated with the survival outcomes of MM. In terms of treatment, the high-risk group showed resistance to conventional drugs such as bortezomib, doxorubicin and immunotherapy. The joint scores generated by the nomogram showed higher clinical benefit than other clinical indicators. The in vitro experiments with cell lines and clinical subjects further provided convincing evidence for our study.Conclusion: We developed and validated the utility of the MM glycolysis-related prognostic model, which provides a new direction for prognosis assessment, treatment options for MM patients

Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps

Yan R, Rodemann T, Wrede B. Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development. 2013;5(4):273-287.For sound localization, the binaural auditory system of a robot needs audio-motor maps, which represent the relationship between certain audio features and the position of the sound source. This mapping is normally learned during an offline calibration in controlled environments, but we show that using computational audiovisual scene analysis (CAVSA), it can be adapted online in free interaction with a number of a priori unknown speakers. CAVSA enables a robot to understand dynamic dialog scenarios, such as the number and position of speakers, as well as who is the current speaker. Our system does not require specific robot motions and thus can work during other tasks. The performance of online-adapted maps is continuously monitored by computing the difference between online-adapted and offline-calibrated maps and also comparing sound localization results with ground truth data (if available). We show that our approach is more robust in multiperson scenarios than the state of the art in terms of learning progress. We also show that our system is able to bootstrap with a randomized audio-motor map and adapt to hardware modifications that induce a change in audio-motor maps

Simple auditory and visual features for human-robot dialog scene analysis

Yan R, Rodemann T, Wrede B. Simple auditory and visual features for human-robot dialog scene analysis. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, NJ: IEEE; 2013: 700-706.This paper presents a system that uses various simple auditory and visual features to achieve human-robot dialog scene analysis. Our scene analysis system is able to learn how many speakers are in the scenario, where the speakers are and who is currently speaking. Speakers are unknown in advance. A visual short-term-memory (STM) helps to memorize persons, even if they disappear from the camera's field of view for a while due to movements of persons or the robot head. In comparison to our previous work, we apply more visual features such as height, color and texture features of different upper body parts, to improve the scene representation performance. We show that our system is able to assign words to corresponding speakers. A speaker is recognized again when he leaves and enters the scene, or changes his position even with a newly appearing person

TANet: A Tiny Plankton Classification Network for Mobile Devices

This paper is devoted to a lightweight convolutional neural network based on the attention mechanism called the tiny attention network (TANet). The TANet consists of three main parts termed as a reduction module, self-attention operation, and group convolution. The reduction module alleviates information loss caused by the pooling operation. The new parameter-free self-attention operation makes the model to focus on learning important parts of images. The group convolution achieves model compression and multibranch fusion. Using the main parts, the proposed network enables efficient plankton classification on mobile devices. The performance of the proposed network is evaluated on the Plankton dataset collected by Oregon State University’s Hatfield Marine Science Center. The results show that TANet outperforms other deep models in speed (31.8 ms per image), size (648 kB, the size of the hard disk space occupied by the model), and accuracy (Top-1 76.5%, Top-5 96.3%)

Advantage of Next-Generation Sequencing in Dynamic Monitoring of Circulating Tumor DNA over Droplet Digital PCR in Cetuximab Treated Colorectal Cancer Patients

Epidermal growth factor receptor (EGFR) blockade resistance is common in the treatment of RAS wide type colorectal cancer (CRC). During the treatment of cetuximab, acquired resistant genomic alterations always occurs earlier than disease progression observed by medical images. Identification of genomic alterations dynamically might have certain clinical significance. Because of the limitation of repeated tissue biopsy, liquid biopsy is increasingly recognized. Droplet digital polymerase chain reaction (ddPCR) is the main detection methods for circulating tumor DNA (ctDNA), however, the application of next-generation sequencing (NGS) for ctDNA detection becomes more and more popular. Here we develop a NGS-based ctDNA assay and evaluated its sensitivity and specificity while using ddPCR as control. These two technologies were both used for genomic alteration detection for the peripheral blood samples from cetuximab-treated colorectal cancer patients dynamically. Fifteen patients were enrolled in this study, including eight males and seven females. The sensitivity and specificity of our NGS assay were 87.5% and 100% respectively, and liner regression analysis comparing variant allele frequency (VAF) revealed high concordance between NGS and ddPCR (R2 = 0.98). NGS actually found more mutation information than ddPCR such as the additional dynamic changes of TP53 which were observed in the disease progression patients. Moreover, the variant allele fraction of TP53 was also found by NGS to be changed along with the clinical efficacy evaluation dynamically during the whole treatment process. In conclusion, our newly developed NGS-based ctDNA assay shows similar performance with ddPCR but have more advantages of its high throughput of multigenetic detection for the dynamic monitoring during the treatment of cetuximab in metastasis CRC patients

ZiBuPiYin recipe protects db/db mice from diabetes-associated cognitive decline through improving multiple pathological changes.

Multiple organ systems, including the brain, which undergoes changes that may increase the risk of cognitive decline, are adversely affected by diabetes mellitus (DM). Here, we demonstrate that type 2 diabetes mellitus (T2DM) db/db mice exhibited hippocampus-dependent memory impairment, which might associate with a reduction in dendritic spine density in the pyramidal neurons of brain, Aβ1-42 deposition in the prefrontal cortex (PFC) and hippocampus, and a decreased expression of neurostructural proteins including microtubule-associated protein (MAP2), a marker of dendrites, and postsynaptic density 95 (PSD95), a marker of excitatory synapses. To investigate the effects of the ZiBuPiYin recipe (ZBPYR), a traditional Chinese medicine recipe, on diabetes-related cognitive decline (DACD), db/db mice received daily administration of ZBPYR over an experimental period of 6 weeks. We then confirmed that ZBPYR rescued learning and memory performance impairments, reversed dendritic spine loss, reduced Aβ1-42 deposition and restored the expression levels of MAP2 and PSD95. The present study also revealed that ZBPYR strengthened brain leptin and insulin signaling and inhibited GSK3β overactivity, which may be the potential mechanism or underlying targets of ZBPYR. These findings conclude that ZBPYR prevents DACD, most likely by improving dendritic spine density and attenuating brain leptin and insulin signaling pathway injury. Our findings provide further evidence for the effects of ZBPYR on DACD

WBSA: Web Service for Bisulfite Sequencing Data Analysis

<div><p>Whole-Genome Bisulfite Sequencing (WGBS) and genome-wide Reduced Representation Bisulfite Sequencing (RRBS) are widely used to study DNA methylation. However, data analysis is complicated, lengthy, and hampered by a lack of seamless analytical pipelines. To address these issues, we developed a convenient, stable, and efficient web service called Web Service for Bisulfite Sequencing Data Analysis (WBSA) to analyze bisulfate sequencing data. WBSA focuses on not only CpG methylation, which is the most common biochemical modification in eukaryotic DNA, but also non-CG methylation, which have been observed in plants, iPS cells, oocytes, neurons and stem cells of human. WBSA comprises three main modules as follows: WGBS data analysis, RRBS data analysis, and differentially methylated region (DMR) identification. The WGBS and RRBS modules execute read mapping, methylation site identification, annotation, and advanced analysis, whereas the DMR module identifies actual DMRs and annotates their correlations to genes. WBSA can be accessed and used without charge either online or local version. WBSA also includes the executables of the Portable Batch System (PBS) and standalone versions that can be downloaded from the website together with the installation instructions. WBSA is available at no charge for academic users at <a href="http://wbsa.big.ac.cn" target="_blank">http://wbsa.big.ac.cn</a>.</p></div